Evaluating the Restrict to MeSH Algorithm

Introduction The goal of the U.S. National Library of Medicine (NLM) is to collect, organize and make available biomedical literature to advance medical science and improve public health. To this end, the Library has developed and maintains a number of online resources. One of the most widely-used is MEDLINE, an online database of references and abstracts from biomedical journals. The ability of patrons to search MEDLINE depends largely on Medical Subject Headings (MeSH), the NLM’s controlled vocabulary thesaurus. MeSH consists of a set of terms arranged into a hierarchical structure. These headings allow for searching the literature at various levels of granularity. MeSH headings are also used for a variety of other purposes, such as for browsing ClinicalTrials.gov, the National Clinical Trials Registry(1). MeSH has broad coverage of topics in a variety of domains. Despite this broad coverage, there are many concepts which appear in the literature but which are not enumerated in MeSH. These are largely concepts that are at a finer level of specificity than that of the MeSH main headings (also called descriptors). To improve the utility of MeSH for indexing the biomedical literature, in 1998 the Restrict to MeSH (RtM) algorithm was devised(2). The algorithm is designed to map entities from many other terminological systems to the MeSH thesaurus, regardless of whether the entities themselves are MeSH headings. Once entities from text are assigned MeSH headings, the semantic information they embody can be leveraged for a variety of purposes, including indexing. The RtM method was evaluated several years ago on a limited basis but it has not yet been evaluated thoroughly. The goals of this project, therefore, were to conduct a qualitative and quantitative evaluation of the results of the algorithm and to recommend steps for its improvement. Although the high-level evaluation is essentially quantitative, it does convey some information about quality of the RtM output. The purpose of the qualitative evaluation is to assess the quality of the individual mappings.